Reducing Sensitivity to NoC Latency in NUCA Caches

Non Uniform Cache Architectures (NUCA) are a novel design paradigm for large last-level on-chip caches which have been introduced to deliver low access latencies in wire-delay dominated environments. Typically, NUCA caches make use of a network-on-chip (NoC) to connect the different sub-banks and the cache controller. This work analyzes how different network parameters, namely hop latency and buffering capacity of routers, affect the overall performance of NUCA-based systems for the single processor case, assuming a reference NUCA organization derived from previous works. This analysis leads to some important design guidelines: first, the sensitivity of the system to the hop latency is very high and router architectures with long pipelines, suitable for throughput-oriented applications, are not adequate; second, limited buffering capacity is sufficient to achieve a good performance level. As a consequence, in this work we propose an alternative NUCA organization based on bank clustering that limits the average number of hops experienced by cache accesses. This organization is better performing in most of the cases and scales better as the cut-through latency increases, thus simplifying the implementation of routers.